<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
	<title>Classic Tokenizer | ElasticSearch 7.7 权威指南中文版</title>
	<meta name="keywords" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <meta name="description" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <!-- Give IE8 a fighting chance -->
    <!--[if lt IE 9]>
    <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
    <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
    <![endif]-->
	<link rel="stylesheet" type="text/css" href="../static/styles.css" />
	<script>
	var _link = 'analysis-classic-tokenizer.html';
    </script>
</head>
<body>
<div class="main-container">
    <section id="content">
        <div class="content-wrapper">
            <section id="guide" lang="zh_cn">
                <div class="container">
                    <div class="row">
                        <div class="col-xs-12 col-sm-8 col-md-8 guide-section">
                            <div style="color:gray; word-break: break-all; font-size:12px;">原英文版地址: <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-classic-tokenizer.html" rel="nofollow" target="_blank">https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-classic-tokenizer.html</a>, 原文档版权归 www.elastic.co 所有<br/>本地英文版地址: <a href="../en/analysis-classic-tokenizer.html" rel="nofollow" target="_blank">../en/analysis-classic-tokenizer.html</a></div>
                        <!-- start body -->
                  <div class="page_header">
<strong>重要</strong>: 此版本不会发布额外的bug修复或文档更新。最新信息请参考 <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html" rel="nofollow">当前版本文档</a>。
</div>
<div id="content">
<div class="breadcrumbs">
<span class="breadcrumb-link"><a href="index.html">Elasticsearch Guide [7.7]</a></span>
»
<span class="breadcrumb-link"><a href="analysis.html">Text analysis</a></span>
»
<span class="breadcrumb-link"><a href="analysis-tokenizers.html">Tokenizer reference</a></span>
»
<span class="breadcrumb-node">Classic Tokenizer</span>
</div>
<div class="navheader">
<span class="prev">
<a href="analysis-chargroup-tokenizer.html">« Char Group Tokenizer</a>
</span>
<span class="next">
<a href="analysis-edgengram-tokenizer.html">Edge n-gram tokenizer »</a>
</span>
</div>
<div class="section">
<div class="titlepage"><div><div>
<h2 class="title">
<a id="analysis-classic-tokenizer"></a>Classic Tokenizer<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenizers/classic-tokenizer.asciidoc">edit</a>
</h2>
</div></div></div>
<p>The <code class="literal">classic</code> tokenizer is a grammar based tokenizer that is good for English
language documents. This tokenizer has heuristics for special treatment of
acronyms, company names, email addresses, and internet host names. However,
these rules don’t always work, and the tokenizer doesn’t work well for most
languages other than English:</p>
<div class="ulist itemizedlist">
<ul class="itemizedlist">
<li class="listitem">
It splits words at most punctuation characters, removing punctuation. However, a
dot that’s not followed by whitespace is considered part of a token.
</li>
<li class="listitem">
It splits words at hyphens, unless there’s a number in the token, in which case
the whole token is interpreted as a product number and is not split.
</li>
<li class="listitem">
It recognizes email addresses and internet hostnames as one token.
</li>
</ul>
</div>
<h3>
<a id="_example_output_9"></a>Example output<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenizers/classic-tokenizer.asciidoc">edit</a>
</h3>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">POST _analyze
{
  "tokenizer": "classic",
  "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/862.console"></div>
<p>The above sentence would produce the following terms:</p>
<div class="pre_wrapper lang-text">
<pre class="programlisting prettyprint lang-text">[ The, 2, QUICK, Brown, Foxes, jumped, over, the, lazy, dog's, bone ]</pre>
</div>
<h3>
<a id="_configuration_10"></a>Configuration<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenizers/classic-tokenizer.asciidoc">edit</a>
</h3>
<p>The <code class="literal">classic</code> tokenizer accepts the following parameters:</p>
<div class="informaltable">
<table border="0" cellpadding="4px">
<colgroup>
<col>
<col>
</colgroup>
<tbody valign="top">
<tr>
<td valign="top">
<p>
<code class="literal">max_token_length</code>
</p>
</td>
<td valign="top">
<p>
The maximum token length. If a token is seen that exceeds this length then
it is split at <code class="literal">max_token_length</code> intervals. Defaults to <code class="literal">255</code>.
</p>
</td>
</tr>
</tbody>
</table>
</div>
<h3>
<a id="_example_configuration_6"></a>Example configuration<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenizers/classic-tokenizer.asciidoc">edit</a>
</h3>
<p>In this example, we configure the <code class="literal">classic</code> tokenizer to have a
<code class="literal">max_token_length</code> of 5 (for demonstration purposes):</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "my_tokenizer"
        }
      },
      "tokenizer": {
        "my_tokenizer": {
          "type": "classic",
          "max_token_length": 5
        }
      }
    }
  }
}

POST my_index/_analyze
{
  "analyzer": "my_analyzer",
  "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/863.console"></div>
<p>The above example produces the following terms:</p>
<div class="pre_wrapper lang-text">
<pre class="programlisting prettyprint lang-text">[ The, 2, QUICK, Brown, Foxes, jumpe, d, over, the, lazy, dog's, bone ]</pre>
</div>
</div>
<div class="navfooter">
<span class="prev">
<a href="analysis-chargroup-tokenizer.html">« Char Group Tokenizer</a>
</span>
<span class="next">
<a href="analysis-edgengram-tokenizer.html">Edge n-gram tokenizer »</a>
</span>
</div>
</div>

                  <!-- end body -->
                        </div>
                        <div class="col-xs-12 col-sm-4 col-md-4" id="right_col">
                        
                        </div>
                    </div>
                </div>
            </section>
        </div>
    </section>
</div>
<script src="../static/cn.js"></script>
</body>
</html>